PENN TREEBANK

Semantic Role Labeling of NomBank Partitives (2025.coling-main)

Challenge:	Existing discriminatory biases in training data can be amplified by models . text corpora exhibit socially problematic biase .
Approach:	They propose a metric to measure gender bias and a regularization loss term to minimize embeddings onto an embeddable subspace that encodes gender.
Outcome:	The proposed method reduces gender bias up to an optimal weight assigned to the loss term, and the model becomes unstable as the perplexity increases.

Copied to clipboard

Adam Meyers, Advait Pravin Savant, John E. Ortega

Bidirectional Masked Self-attention and N-gram Span Attention for Constituency Parsing (2023.findings-emnlp)

Challenge:	Semantic role labeling (SRL) is a way to represent semantic concepts via labeled predicate/argument pairs.
Approach:	They describe a semantic role labeling task that uses a set of predicate/argument pairs to represent semantic concepts.
Outcome:	The highest scoring system achieves an F1 of 91.74% using “gold” parses from the Penn Treebank and 91.12% when using the Berkeley Neural parser.

Copied to clipboard

Soohyeong Kim, Whanhee Cho, Minji Kim, Yong Choi

Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context (P18-1)

Challenge:	Existing attention mechanisms for constituency parsing lack directional information needed to form sentence spans.
Approach:	They propose a bidirectional masked and N-gram span Attention model which captures the explicit dependencies between each word and enhances the representation of the output span vectors.
Outcome:	The proposed model achieves state-of-the-art performance on the Penn Treebank and Chinese Penn TreeBank datasets with F1 scores of 96.47 and 94.15 respectively.

Copied to clipboard

Urvashi Khandelwal, He He, Peng Qi, Dan Jurafsky

Penn-Helsinki Parsed Corpus of Early Modern English: First Parsing Results and Analysis (2022.findings-naacl)

Challenge:	Recent studies have shed light on the information encoded by long-term memory networks.
Approach:	They propose to use a neural caching model to model the role of context in an LSTM LM . they analyze the increase in perplexity when prior context words are shuffled, replaced, or dropped .
Outcome:	The proposed model is highly sensitive to the order of words within the most recent sentence, but ignores word order in the long-range context, suggesting the distant past is modeled only as a rough semantic field or topic.

Copied to clipboard

Seth Kulick, Neville Ryant, Beatrice Santorini

Rethinking Self-Attention: Towards Interpretability in Neural Parsing (2020.findings-emnlp)

Challenge:	PPCEME has a large set of function tags and is difficult to parse . authors present results for PPceME using a modified version of the Berkeley Neural Parser .
Approach:	They propose to use a modified version of the Berkeley Neural Parser to parse PPCEME using function tags.
Outcome:	The proposed parser will be used to parse Early English Books Online, a 1.5 billion word corpus.

Copied to clipboard

Khalil Mrini, Franck Dernoncourt, Quan Hung Tran, Trung Bui, Walter Chang, Ndapa Nakashole

Linear-time Constituency Parsing with RNNs and Dynamic Programming (P18-2)

Challenge:	Recent work shows that attention mechanisms provide arguably explainable attention distributions that can help to interpret predictions.
Approach:	They propose a new self-attention layer where attention heads represent labels.
Outcome:	The proposed model obtains state-of-the-art results on the Penn Treebank and Chinese Treebank.

Copied to clipboard

Juneki Hong, Liang Huang

Multimodality

Challenge:	Existing span-based constituency parsers are too slow for longer sentences and for applications beyond sentence boundaries.
Approach:	They propose a linear-time constituency parser with RNNs and dynamic programming using graph-structured stack and beam search.
Outcome:	The proposed parser is faster for long sentences and faster for discourse parsing.

The ACoLi CoNLL Libraries: Beyond Tab-Separated Values (L18-1)

Copied to clipboard

Christian Chiarcos, Niko Schenk

An Argument for Symmetric Coordination from Dependency Length Minimization: A Replication Study (2024.lrec-main)

Challenge:	a new set of Java archives facilitates advanced manipulations of corpora annotated in TSV formats.
Approach:	They propose to use Java archives to facilitate advanced manipulations of corpora annotated in TSV formats.
Outcome:	The proposed libraries support all members of the CoNLL format family.

Copied to clipboard

Adam Przepiórkowski, Magdalena Borysiak, Adam Głowacki

Challenge:	a growing number of researchers are focusing on replication studies, but this is not the only reason.
Approach:	They replicate a result reported in a recent ACL paper that suggests left conjuncts are shorter in English coordinate structures.
Outcome:	The proposed results provide an argument for the symmetric structure of coordination in English coordinate structures.

Non-Projective Dependency Parsing with Non-Local Transitions (N18-2)

Copied to clipboard

Daniel Fernández-González, Carlos Gómez-Rodríguez

Improved Language Modeling by Decoding the Past (P19-1)

Challenge:	Existing greedy transition-based parsers are prone to error propagation when creating arcs involving multiple transitions.
Approach:	They propose a greedy transition-based parser that introduces non-local transitions that create arcs involving nodes to the left of the current focus positions.
Outcome:	The proposed system outperforms the original version and achieves the best accuracy on the Stanford Dependencies conversion of the Penn Treebank among greedy transition-based parsers.

Copied to clipboard

Siddhartha Brahma

CRYPTOGRU: Low Latency Privacy-Preserving Text Analysis With GRU (2021.emnlp-main)

Challenge:	Existing methods to improve language modeling performance are based on regularized LSTMs with a large number of parameters and training time.
Approach:	They propose a method that decodes the last token in context using the predicted distribution of the next token.
Outcome:	The proposed method improves perplexity on the Penn Treebank dataset by 1.8 points and 2.3 points on the WikiText-2 datasets.

Copied to clipboard

Bo Feng, Qian Lou, Lei Jiang, Geoffrey Fox

Responsible & Trustworthy NLP

Challenge:	Existing approaches to combine HE and GC in RNNs suffer from long inference latency due to the slow activation functions.
Approach:	They propose a hybrid structure of HE and GC gated recurrent unit network, for low-latency secure inferences.
Outcome:	The proposed structure improves the secure inference latency by up to 138 over one of the state-of-the-art secure networks on the Penn Treebank dataset.

Valency-Augmented Dependency Parsing (D18-1)

Copied to clipboard

Tianze Shi, Lillian Lee

Unsupervised Learning of Syntactic Structure with Invertible Neural Projections (D18-1)

Challenge:	valency analysis is a complex task that requires a large number of subcategorizations, such as the number and types of syntactic dependents.
Approach:	They propose a parsing approach that explicitly models the number and types of syntactic dependents as valency patterns and a probabilistic model for tagging them.
Outcome:	The proposed approach outperforms the state-of-the-art labeled attachment score on 53 treebanks representing 41 languages and outperformed the previous state- of-the art labeles by 0.7.

Copied to clipboard

Junxian He, Graham Neubig, Taylor Berg-Kirkpatrick

Responsible & Trustworthy NLP

Challenge:	Unsupervised learning of syntactic structure is typically performed using generative models with discrete latent variables and multinomial parameters.
Approach:	They propose a generative model that jointly learns discrete syntactic structure and continuous word representations in an unsupervised fashion by cascading an invertible neural network with a structured generative prior.
Outcome:	The proposed model outperforms state-of-the-art models on part-of speech (POS) induction and unsupervised dependency parsing without gold POS annotation.

Unsupervised Induction of Linguistic Categories with Records of Reading, Speaking, and Writing (N18-1)

Copied to clipboard

Maria Barrett, Ana Valeria González-Garduño, Lea Frermann, Anders Søgaard

Responsible & Trustworthy NLP

Challenge:	a few researchers have shown that data traces from human processing can be used to improve NLP models.
Approach:	They propose to use data readily available for most languages to improve unsupervised induction . they find that english unsupervised POS induction achieves an error reduction of 1.5% .
Outcome:	The proposed model improves on Ontonotes domains with a word embeddings.

Neural Combinatory Constituency Parsing (2021.findings-acl)

Copied to clipboard

Zhousi Chen, Longtu Zhang, Aizhan Imankulova, Mamoru Komachi

Learned Incremental Representations for Parsing (2022.acl-long)

Challenge:	Existing approaches to constituency parsing are based on symbolic engineering, but they are simplified by their adaptive distributed representation.
Approach:	They propose two fast combinatory models for constituency parsing: binary and multibranching.
Outcome:	The proposed models achieve an F1 score of 92.54 on Penn Treebank, speeding at 1327.2 sents/sec.

Copied to clipboard

Nikita Kitaev, Thomas Lu, Dan Klein

Representation Learning

Challenge:	a new syntactic representation that commits to syntakic choices is proposed for humans . we use a system that uses only incremental processing of a prefix to predict the word in a sentence .
Approach:	They propose a syntactic representation that commits to syntakic choices incrementally . they say the system can achieve 93.72 F1 on the Penn Treebank with as few as 5 bits per word .
Outcome:	The proposed representation achieves 93.72 F1 on the Penn Treebank with as few as 5 bits per word . the analysis of the representations shows they have interpretable features and deferred resolution of syntactic ambiguities.

Head-Driven Phrase Structure Grammar Parsing on Penn Treebank (P19-1)

Copied to clipboard

Junru Zhou, Hai Zhao

Constituency Parsing with a Self-Attentive Encoder (P18-1)

Challenge:	Head-driven phrase structure grammars have a uniform formalism representing rich contextual syntactic and even semantic meanings.
Approach:	They propose to integrate constituent and dependency formal representations into head-driven phrase structure.
Outcome:	The proposed parser achieves state-of-the-art performance on Penn Treebank and Chinese Penn TreeBank.

Copied to clipboard

Nikita Kitaev, Dan Klein

Let’s do it “again”: A First Computational Approach to Detecting Adverbial Presupposition Triggers (P18-1)

Challenge:	Recent work on LSTM encoders based on recurrent neural networks has led to improvements in constituency parsing accuracy.
Approach:	They propose to replace an LSTM encoder with a self-attentive architecture to improve a discriminative constituency parser.
Outcome:	The proposed model outperforms the previous best-published results on 8 of the 9 languages in the SPMRL dataset.

Copied to clipboard

Andre Cianflone, Yulan Feng, Jad Kabbara, Jackie Chi Kit Cheung

Challenge:	a novel task of predicting adverbial presupposition triggers is useful for natural language generation . a focus is on a new attention mechanism for predicting presuposition trigger .
Approach:	They propose a new attention mechanism for predicting adverbial presupposition triggers . they propose to augment a baseline neural network without additional trainable parameters .
Outcome:	The proposed model outperforms baseline models in predicting adverbial presupposition triggers.

Bridging Pre-trained Language Models and Hand-crafted Features for Unsupervised POS Tagging (2022.findings-acl)

Copied to clipboard

Houquan Zhou, Yang Li, Zhenghua Li, Min Zhang

Transformer-XL: Attentive Language Models beyond a Fixed-Length Context (P19-1)

Challenge:	Large-scale pre-trained language models (PLMs) have made extraordinary progress in most NLP tasks, but they fail to achieve state-of-the-art (SOTA) performance.
Approach:	They propose a Guassian HMM variant for unsupervised POS tagging that incorporates contexualized word representations into the decoder.
Outcome:	The proposed model outperforms state-of-the-art models on Penn Treebank and multilingual Universal Dependencies treebank v2.0.

Copied to clipboard

Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc Le, Ruslan Salakhutdinov

BERT-Proof Syntactic Structures: Investigating Errors in Discontinuous Constituency Parsing (2021.findings-acl)

Challenge:	Term memory networks (RNNs) are difficult to optimize due to gradient vanishing and explosion.
Approach:	They propose a neural architecture Transformer-XL that enables learning dependency beyond a fixed length without disrupting temporal coherence.
Outcome:	The proposed method improves state-of-the-art performance on short and long sequences and generates coherent, novel text articles with thousands of tokens.

Copied to clipboard

Maximin Coavoux

A Corpus-based Syntactic Analysis of Two-termed Unlike Coordination (2021.findings-emnlp)

Challenge:	Recent results show that pretrained language models can be used for many tasks with high accuracy and high performance.
Approach:	They propose two methods for automatically analysing discontinuous parsers' errors.
Outcome:	The proposed methods characterize errors of a state-of-the-art transition-based discontinuous parser and provide an overview of the contribution of BERT to this task.

Copied to clipboard

Julie Kallini, Christiane Fellbaum

Women’s Syntactic Resilience and Men’s Grammatical Luck: Gender-Bias in Part-of-Speech Tagging and Dependency Parsing (P19-1)

Challenge:	a phenomenon of language that conjoins two or more terms or phrases using a coordinating conjunction is still largely elusive and widely debated amongst linguists.
Approach:	They propose to use a computational corpus-based approach to study two-termed unlike coordinations where the two conjuncts of the coordination phrase form valid constituents but have distinct categories.
Outcome:	The proposed analysis shows that the two conjuncts within unlike coordinations display different properties based on their position, supporting an antisymmetric view of the structure of coordination.

Copied to clipboard

Aparna Garimella, Carmen Banea, Dirk Hovy, Rada Mihalcea

Cost-Effective Discourse Annotation in the Prague Czech–English Dependency Treebank (2024.lrec-main)

Challenge:	linguistic studies have shown the prevalence of various lexical and grammatical patterns in texts authored by a person of a particular gender, but models for part-of-speech tagging and dependency parsing have not adapted to account for these differences.
Approach:	They annotate the Wall Street Journal part of the Penn Treebank with the gender information of the articles’ authors and build taggers and parsers trained on this data.
Outcome:	The proposed model can account for gendered differences in syntactic tasks and highlight future venues for developing more accurate taggers and parsers.

Copied to clipboard

Jiří Mírovský, Pavlína Synková, Lucie Polakova, Marie Paclíková

Parsing All: Syntax and Semantics, Dependencies and Spans (2020.findings-emnlp)

Challenge:	a method for obtaining a high-quality annotation of explicit discourse relations is a resource-demanding task.
Approach:	They propose a method for obtaining a high-quality annotation of explicit discourse relations in the Czech part of the Prague Czech–English Dependency Treebank.
Outcome:	The proposed method solves the problem of identifying discrepancies between the annotations in the Czech part of the Penn Treebank.

Copied to clipboard

Junru Zhou, Zuchao Li, Hai Zhao

Experimenting with Power Divergences for Language Modeling (D19-1)

Challenge:	Syntactic and semantic structures are key linguistic contextual clues, but few studies have explored how they can be used to improve syntactical parsing.
Approach:	They propose a syntactic and semantic parsing model which integrates syntaktic information in the encoder of neural network and benefits from two representation formalisms in a uniform way.
Outcome:	The proposed model achieves state-of-the-art or competitive results on both span and dependency representations and on Penn Treebank.

Copied to clipboard

Matthieu Labeau, Shay B. Cohen

Direct Output Connection for a High-Rank Language Model (D18-1)

Challenge:	Language models are an important component in many NLP tasks, where they provide prior knowledge on the language used.
Approach:	They propose to use power divergences to prioritize learning on frequent or rare words . they use a sample-based objective to approximate a softmax and noise-constrained estimate .
Outcome:	The proposed power divergences can be used to prioritize learning on the frequent or rare words and lead to general performance improvements.

Copied to clipboard

Sho Takase, Jun Suzuki, Masaaki Nagata

Probing for Constituency Structure in Neural Language Models (2022.findings-emnlp)

Challenge:	Neural network language models have played a central role in recent natural language processing advances.
Approach:	They propose a recurrent neural network language model that combines probability distributions computed from a final RNN layer and also middle layers.
Outcome:	The proposed language model achieves the best score on the Penn Treebank and WikiText-2 benchmark datasets.

Copied to clipboard

David Arps, Younes Samih, Laura Kallmeyer, Hassan Sajjad

NNE: A Dataset for Nested Named Entity Recognition in English Newswire (P19-1)

Challenge:	Using standard probing techniques, we examine whether contextual neural language models implicitly learn syntactic structure.
Approach:	They investigate to which extent contextual neural language models implicitly learn syntactic structure.
Outcome:	The proposed model is able to represent constituents of different categories within the neuron activations of a LM such as RoBERTa with high performance even on manipulated data.

Copied to clipboard

Nicky Ringland, Xiang Dai, Ben Hachey, Sarvnaz Karimi, Cecile Paris, James R. Curran

Information Extraction & Text Mining

Challenge:	Named entity recognition (NER) is widely used in downstream tasks but most tools focus on flat mention structure over coarse schemas.
Approach:	They describe a fine-grained, nested named entity dataset over the Wall Street Journal portion of the Penn Treebank.
Outcome:	The proposed dataset comprises 279,795 mentions of 114 entity types with up to 6 layers of nesting.

PTB Graph Parsing with Tree Approximation (P19-1)

Copied to clipboard

Yoshihide Kato, Shigeki Matsubara

Structured Data in NLP

Challenge:	a new method for nonlocal dependency identification is proposed for the Penn Treebank (PTB) syntactic structures are represented as graphs due to nonlocal dependencies .
Approach:	They propose a method that approximates PTB graph-structured representations by trees . their method reduces nonlocal dependency identification and constituency parsing into single parser .
Outcome:	The proposed method outperforms the previous methods in nonlocal dependency identification.

Better Exploiting Latent Variables in Text Modeling (P19-1)

Copied to clipboard

Canasai Kruengkrai

Challenge:	Consistent gains in performance on two datasets, Penn Treebank and Yahoo, indicate the generalizability of our method.
Approach:	They propose a method to exploit latent variables through hidden state averaging by sampling latent variable multiple times at a gradient step.
Outcome:	The proposed method shows consistent gains on two datasets showing that it is generalizable.

Language Modeling with a General Second-Order RNN (2020.lrec-1)

Copied to clipboard

Diego Maupomé, Marie-Jean Meurs

Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach (2020.acl-main)

Challenge:	a number of RNNs update their state as the input sequence is processed . second-order RNN architectures show promising performance in language modeling .
Approach:	They propose a second-order RNN architecture that generalizes existing ones . they use a Penn Treebank dataset to analyze how their different components affect performance .
Outcome:	The proposed architecture generalizes existing RNNs on a Penn Treebank dataset . it shows that removing the first-order terms does not hinder performance .

Copied to clipboard

Wenyu Du, Zhouhan Lin, Yikang Shen, Timothy J. O’Donnell, Yoshua Bengio, Yue Zhang

Improving Low Compute Language Modeling with In-Domain Embedding Initialisation (2020.emnlp-main)

Challenge:	incorporating syntactic structure into language models has been a challenge since the 1990s.
Approach:	They propose to use syntactic information to integrate syntastic structure into neural language models by providing ground truth parse trees as additional training signals.
Outcome:	The proposed model achieves lower perplexity and better quality when ground truth parse trees are provided as training signals.

Copied to clipboard

Charles Welch, Rada Mihalcea, Jonathan K. Kummerfeld